Haptic interfaces: Hardware, software and human performance

Virtual environments are computer-generated synthetic environments with which a human user can interact to perform a wide variety of perceptual and motor tasks. At present, most of the virtual environment systems engage only the visual and auditory senses, and not the haptic sensorimotor system that conveys the sense of touch and feel of objects in the environment. Computer keyboards, mice, and trackballs constitute relatively simple haptic interfaces. Gloves and exoskeletons that track hand postures have more interaction capabilities and are available in the market. Although desktop and wearable force-reflecting devices have been built and implemented in research laboratories, the current capabilities of such devices are quite limited. To realize the full promise of virtual environments and teleoperation of remote systems, further developments of haptic interfaces are critical. In this paper, the status and research needs in human haptics, technology development and interactions between the two are described. In particular, the excellent performance characteristics of Phantom, a haptic interface recently developed at MIT, are highlighted. Realistic sensations of single point of contact interactions with objects of variable geometry (e.g., smooth, textured, polyhedral) and material properties (e.g., friction, impedance) in the context of a variety of tasks (e.g., needle biopsy, switch panels) achieved through this device are described and the associated issues in haptic rendering are discussed

Lump detection with a gelsight sensor

A GelSight sensor is a tactile sensing device comprising a clear elastomeric pad covered with a reflective membrane, coupled with optics to measure the membrane's deformations. When the pad is pressed against an object's surface, the membrane changes shape in accord with mechanical and geometrical properties of the object. Since soft tissue is more compliant than hard tissue, one can detect an embedded lump by pressing the GelSight pad against the tissue surface and observing the hump that forms over the lump. We tested this system's sensitivity by constructing phantoms of soft rubber with hard embedded lumps. The system is quite sensitive; for example it could detect a 2mm lump at a depth of 5mm. The sensor was more sensitive than previous tactile lump detectors. It was also better than human observers using their fingertips. Such a capability could help in tumor screening, and could augment the sensory information available in telemedicine or minimally invasive surgery.National Science Foundation (U.S.) (Grant 6922551

Shape-independent hardness estimation using deep learning and a GelSight tactile sensor

Hardness is among the most important attributes of an object that humans learn about through touch. However, approaches for robots to estimate hardness are limited, due to the lack of information provided by current tactile sensors. In this work, we address these limitations by introducing a novel method for hardness estimation, based on the GelSight tactile sensor, and the method does not require accurate control of contact conditions or the shape of objects. A GelSight has a soft contact interface, and provides high resolution tactile images of contact geometry, as well as contact force and slip conditions. In this paper, we try to use the sensor to measure hardness of objects with multiple shapes, under a loosely controlled contact condition. The contact is made manually or by a robot hand, while the force and trajectory are unknown and uneven. We analyze the data using a deep constitutional (and recurrent) neural network. Experiments show that the neural net model can estimate the hardness of objects with different shapes and hardness ranging from 8 to 87 in Shore 00 scale

Expertise and Trust-Aware Social Web Service Recommendation

With the increasing number of Web services, the personalized recommendation of Web services has become more and more important. Fortunately, the social network popularity nowadays brings a good alternative for social recommendation to avoid the data sparsity problem that is not treated very well in the collaborative filtering approach. Since the social network provides a big data about the users, the trust concept has become necessary to filter this abundance and to foster the successful interactions between the users. In this paper, we firstly propose a trusted friend detection mechanism in a social network. The dynamic of the users’ interactions over time and the similarity of their interests have been considered. Secondly, we propose a Web service social recommendation mechanism which considers the expertise of the trusted friends according to their past invocation histories and the active user’s query. The experiments of each mechanism produced satisfactory results

Localization and Manipulation of Small Parts Using GelSight Tactile Sensing

Robust manipulation and insertion of small parts can be challenging because of the small tolerances typically involved. The key to robust control of these kinds of manipulation interactions is accurate tracking and control of the parts involved. Typically, this is accomplished using visual servoing or force-based control. However, these approaches have drawbacks. Instead, we propose a new approach that uses tactile sensing to accurately localize the pose of a part grasped in the robot hand. Using a feature-based matching technique in conjunction with a newly developed tactile sensing technology known as GelSight that has much higher resolution than competing methods, we synthesize high-resolution height maps of object surfaces. As a result of these high-resolution tactile maps, we are able to localize small parts held in a robot hand very accurately. We quantify localization accuracy in benchtop experiments and experimentally demonstrate the practicality of the approach in the context of a small parts insertion problem.National Science Foundation (U.S.) (NSF Grant No. 1017862)United States. National Aeronautics and Space Administration (NASA under Grant No. NNX13AQ85G)United States. Office of Naval Research (ONR Grant No. N000141410047

Sensory Communication

Contains table of contents for Section 2, an introduction, reports on nine research projects and a list of publications.National Institutes of Health Grant 5 R01 DC00117National Institutes of Health Grant 2 R01 DC00270National Institutes of Health Grant 1 P01 DC00361National Institutes of Health Grant 2 R01 DC00100National Institutes of Health Grant FV00428National Institutes of Health Grant 5 R01 DC00126U.S. Air Force - Office of Scientific Research Grant AFOSR 90-200U.S. Navy - Office of Naval Research Grant N00014-90-J-1935National Institutes of Health Grant 5 R29 DC0062

Beaming into the Rat World: Enabling Real-Time Interaction between Rat and Human Each at Their Own Scale

Immersive virtual reality (IVR) typically generates the illusion in participants that they are in the displayed virtual scene where they can experience and interact in events as if they were really happening. Teleoperator (TO) systems place people at a remote physical destination embodied as a robotic device, and where typically participants have the sensation of being at the destination, with the ability to interact with entities there. In this paper, we show how to combine IVR and TO to allow a new class of application. The participant in the IVR is represented in the destination by a physical robot (TO) and simultaneously the remote place and entities within it are represented to the participant in the IVR. Hence, the IVR participant has a normal virtual reality experience, but where his or her actions and behaviour control the remote robot and can therefore have physical consequences. Here, we show how such a system can be deployed to allow a human and a rat to operate together, but the human interacting with the rat on a human scale, and the rat interacting with the human on the rat scale. The human is represented in a rat arena by a small robot that is slaved to the human’s movements, whereas the tracked rat is represented to the human in the virtual reality by a humanoid avatar. We describe the system and also a study that was designed to test whether humans can successfully play a game with the rat. The results show that the system functioned well and that the humans were able to interact with the rat to fulfil the tasks of the game. This system opens up the possibility of new applications in the life sciences involving participant observation of and interaction with animals but at human scale

Sensory Communication

Contains table of contents for Section 2, an introduction, reports on ten research projects and a list of publications.National Institutes of Health Grant 5 R01 DC00117National Institutes of Health Grant 5 R01 DC00270National Institutes of Health Grant 5 P01 DC00361National Institutes of Health Grant 2 R01 DC00100National Institutes of Health Grant 7 R29 DC00428National Institutes of Health Grant 2 R01 DC00126U.S. Air Force - Office of Scientific Research Grant AFOSR 90-0200U.S. Navy - Office of Naval Research Grant N00014-90-J-1935National Institutes of Health Grant 5 R29 DC00625U.S. Navy - Office of Naval Research Grant N00014-91-J-145